Neural networks are computational systems that permit computers to essentially function in a manner analogous to that of the human brain. Neural networks do not utilize the traditional digital model of manipulating 0's and 1's. Instead, neural networks create connections between processing elements, which are equivalent to neurons of a human brain. Neural networks are thus based on various electronic circuits that are modeled on human nerve cells (i.e., neurons). Generally, a neural network is an information-processing network, which is inspired by the manner in which a human brain performs a particular task or function of interest. Computational or artificial neural networks are thus inspired by biological neural systems. The elementary building block of biological neural systems is of course the neuron, the modifiable connections between the neurons, and the topology of the network.
In general, artificial neural networks are systems composed of many nonlinear computational elements operating in parallel and arranged in patterns reminiscent of biological neural nets. The computational elements, or nodes, are connected via variable weights that are typically adapted during use to improve performance. Thus, in solving a problem, neural net models can explore many competing hypothesis simultaneously using massively parallel nets composed of many computational elements connected by links with variable weights.
In a neural network, “neuron-like” nodes can output a signal based on the sum of their inputs, the output being the result of an activation function. In a neural network, there exists a plurality of connections, which are electrically coupled among a plurality of neurons. The connections serve as communication bridges among of a plurality of neurons coupled thereto. A network of such neuron-like nodes has the ability to process information in a variety of useful ways. By adjusting the connection values between neurons in a network, one can match certain inputs with desired outputs.
Neural networks that have been developed to date are largely software-based. A true neural network (e.g., the human brain) is massively parallel (and therefore very fast computationally) and very adaptable. For example, half of a human brain can suffer a lesion early in its development and not seriously affect its performance. Software simulations are slow because during the learning phase a standard computer must serially calculate connection strengths. When the networks get larger (and therefore more powerful and useful), the computational time becomes enormous. For example, networks with 10,000 connections can easily overwhelm a computer. In comparison, the human brain has about 100 billion neurons, each of which can be connected to about 5,000 other neurons. On the other hand, if a network is trained to perform a specific task, perhaps taking many days or months to train, the final useful result can be etched onto a piece of silicon and also mass-produced.
A number of software simulations of neural networks have been developed. Because software simulations are performed on conventional sequential computers, however, they do not take advantage of the inherent parallelism of neural network architectures. Consequently, they are relatively slow.
The implementation of neural network systems has lagged somewhat behind their theoretical potential due to the difficulties in building neural network hardware. This is primarily because of the large numbers of neurons and weighted connections required. The emulation of even of the simplest biological nervous systems would require neurons and connections numbering in the millions. Due to the difficulties in building such highly interconnected processors, the currently available neural network hardware systems have not approached this level of complexity. Another disadvantage of hardware systems is that they typically are often custom designed and built to implement one particular neural network architecture and are not easily, if at all, reconfigurable to implement different architectures. A true physical neural network (i.e., artificial neural network) chip, for example, has not yet been designed and successfully implemented.
The problem with pure hardware implementation of a neural network with technology as it exists today, is the inability to physically form a great number of connections and neurons. On-chip learning can exist, but the size of the network would be limited by digital processing methods and associated electronic circuitry. One of the difficulties in creating true physical neural networks lies in the highly complex manner in which a physical neural network must be designed and built. It is believed that solutions to creating a true physical and artificial neural network lie in the use of nanotechnology and the implementation of analog variable connections.
The term “Nanotechnology” generally refers to nanometer-scale manufacturing processes, materials and devices, as associated with, for example, nanometer-scale lithography and nanometer-scale information storage. Nanometer-scale components find utility in a wide variety of fields, particularly in the fabrication of microelectrical and microelectromechanical systems (commonly referred to as “MEMS”). Microelectrical nano-sized components include transistors, resistors, capacitors and other nano-integrated circuit components. MEMS devices include, for example, micro-sensors, micro-actuators, micro-instruments, micro-optics, and the like.
Present chip technology is also limiting when wires need to be crossed on a chip. For the most part, the design of a computer chip is limited to two dimensions. Each time a circuit must cross another circuit, another layer must be added to the chip. This increases the cost and decreases the speed of the resulting chip. A number of alternatives to standard silicon based complementary metal oxide semiconductor (“CMOS”) devices have been proposed. The common goal is to produce logic devices on a nanometer scale. Such dimensions are more commonly associated with molecules than integrated circuits.
Integrated circuits and electrical components thereof, which can be produced at a molecular and nanometer scale, include devices such as carbon nanotubes and nanowires, which essentially are nanoscale conductors (“nanoconductors”). Nanocondcutors are tiny conductive tubes (i.e., hollow) or wires (i.e., solid) with a very small size scale (e.g., 1.0–100 nanometers in diameter and hundreds of microns in length). Their structure and fabrication have been widely reported and are well known in the art. Carbon nanotubes, for example, exhibit a unique atomic arrangement, and possess useful physical properties such as one-dimensional electrical behavior, quantum conductance, and ballistic electron transport.
Carbon nanotubes are among the smallest dimensioned nanotube materials with a generally high aspect ratio and small diameter. High-quality single-walled carbon nanotubes can be grown as randomly oriented, needle-like or spaghetti-like tangled tubules. They can be grown by a number of fabrication methods, including chemical vapor deposition (CVD), laser ablation or electric arc growth. Carbon nanotubes can be grown on a substrate by catalytic decomposition of hydrocarbon containing precursors such as ethylene, methane, or benzene. Nucleation layers, such as thin coatings of Ni, Co, or Fe are often intentionally added onto the substrate surface in order to nucleate a multiplicity of isolated nanotubes. Carbon nanotubes can also be nucleated and grown on a substrate without a metal nucleating layer by using a precursor including one or more of these metal atoms. Semiconductor nanowires can be grown on substrates by similar processes.
Based on the foregoing, it is believed that a physical neural network which incorporates nanotechnology is a solution to the problems encountered by prior art neural network solutions. It is believed that a true physical neural network can be designed and constructed without relying on computer simulations for training, or relying on standard digital (binary) memory to store connections strengths. Additionally, it is believed that a variable resistor apparatus can be constructed based on nanotechnology and utilized either as an individual component for variable resistance purposes, or in association with physical neural networks, including artificial neurons and components thereof as described herein.